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Signalling System 



Summary 

The benefits of captioning systems for the hard of hearing are wed understood; 
However caption systems can be costly and they often obscure part of the picture. 
This problem is particularly acute in cinemas. This invention makes use of standard 
mobile phones'to receive, synchronise and display caption text. 

Problem 

A significant proportion of the population with hearing difficulties benefit from captions 
on video images such as TV broadcasts, video tapes, DVD and movies. However, 
there are a number of problems with current solutions. With open captions systems 
where the text is displayed on screen it obscures part of the image. This presents a 
particular problem with cinema where there is a reluctance for this to happen with 
general audiences. It is also not possible to personalise content eg different 
languages. There are closed caption systems where the text is displayed on a 
separate screen. The problem with such a solution is that it adds additional cost and 
complexity and such systems have had poor take up in cinemas for this reason. 

invention 

This invention uses widely available phone and hand-held computers as the caption 
display device. The invention relates to how these can be used to successfully display 
captions that are accurately synchronised to the video/picture content 

Caption text 

The caption text can be transmitted to the hand-held device either in advance of the 
programme content happening or simultaneously with the content. 

If it is done in advance this can be achieved by well recognised methods such as 
transfer by memory card, cable, infra-red link or over a radio communications network 
such as the mobile phone network or a wireless LAN. The caption text can be stored 
on a database and made available via the Internet for download by the methods 
descirbed above.The caption text would contain timecode related information that 
would be triggered during the programme using timer in the phone and/or with some 
form of external synchronisation. 

Real-time delivery could be by similar methods but could include transmission of the 
text information in the audio of the content using a range of acoustic transmission 
techniques such as WO 98/32248. other techniques include: simple amplitude 
and frequency modulation of tones, echo modulation, critical band encoding. Masking 
of the data sound can be performed using a variety of pyscho-acoustic approaches. At 
the required data rate for real-time transmission it is likely that it may be obtrusive to 
listeners with normal hearing therefore performing the transfer in advance is 
advantageous. 




Synchronisation 

Synchronisation of the text is important to ensure that it occurs at the right point In the 
programme content. It is particularly Important when the captions are loaded on to the 
hand-held device in advance. The captions could be started by the user pressing a 
button and button based control could be provided to adjust the synchronisation during 
the programme. Once initiated synchronisation would be maintained by using a timer 
in the device that triggers the captions based on related timing information associated 
with the text- This has the advantage of being simple but requires obvious cues that a 
deaf person can interpret It also suffers from potential clock drift between the source 
and the timer in the device. A better solution is to provide cues directly to the device. 
This can be done by embedding synchronisation cues in the audio of the programme 
content. These can be decoded by using a microphone and processor found in digital 
mobile phones. Embedding cues only, rather that the text requires a significantly 
lower data rate and can be inserted and convenient points where the audio will mask 
them. The techniques for this are similar to those described above for transmission of 
the caption text. 

The synchronisation codes used can either be the same code repeated whenever the 
synchronisation is required or it can be a unique code at each synchronisation point. 
The advantage of having a unique code at the synchronisation point is that a user who 
enters the film or requires the captions only at certain points (for example a user who 
can only rarely requires the language translation captions) can start captioning at any 
point during the film. This is completely in the control of the user in this method. 

It is envisaged that the synchronisation codes will only be required every minute or so 
in a preferred deployment This results in very low obtrusiveness of the 
synchronisation codes. For lower power consumption in the hand held device it is 
possible to set up the hand held device so that it knows when to expect the 
synchronisation code approximately if it has downloaded the text and synchronisation 
choreography prior to the code being transmitted. For example, when a user enters a 
cinema they start the captioning option on their hand held device. The hand held 
device listens continuously for a synchronisation code until it receives one. The hand 
held device can then start the captioning from the point which corresponds to that 
particular synchronisation code (either the beginning or if it is a unique code then the 
unique point in the film that that corresponds to). The hand held device knows 
approximately when to expect the next synchronisation code (from the synchronisation 
choreography previously downloaded) and so it can save power by switching off the 
listening mode until a time window around the expected synchronisation point This 
time window can be defined by the manufacturer of the hand held device or the user. 
This allows the hand held device to check its synchronisation with the film every 
minute or so in a low power mode. 

To those skilled in the art it is possible to extend this text captioning to audio 
captioning. For example, the portion of the population who are visually impaired would 
benefit from additional information about what is happening on the screen. This cannot 
be delivered through a textual interface but could be delivered through the audio 
channel of the hand held device. The synchronisation and source of the audio 
captioning would be the same as that for the text captioning and can be downloaded 
prior to viewing. The user would then put earphones on during the show so that they 
can receive the extra information. This could also be used to transmit a different 



language if the film has been dubbed so that each listener in the cinema could have 
their preferred language playing. 

Other solutions for cues using a radio network such as a Wireless LAN or Infra-red 
could also be used. Cues could also be taken from the video content. 

Display 

♦TI 1 ! °°" ,d a '!° be P® rformed usin 3 'head-up display* principles and equipment 
to enable the viewer to see the content and the captions simultaneously. 

Personalisation 

22J? a<Jva . n 59. e J s ° f " sin . g a hand-held device Is that the caption content can be 

fr222J21 an * J? !f' baS ' S - For examp,e different languages or additional 
information eg the directors comments or the names of the actors in a film. 

Applications 

p^Hcp?eSSons nC,Ude Capti ° ning f ° r radio ' W « recorded media . theatre. 

The contents of the following related applications are incorporated 
herein by reference: 

WO 98/32248 
WO 01/10065 

International Application No. PCT/GB01/05300 

International Application No. PCT/GBO 1/05306 



-3- 





Movie Screen 



3. Audio played back 
through conventional 
speakers 
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5, The triggers are used 
to display the correct 
caption on the screen 
of the phone. 





1 . Captions with timecodes 
or similar reference 
downloaded to the phone 
for example by the mobile 
phone network. 



2. Audio triggers 
embedded in the 
sound track of 
the movie 



4. Audio triggers 
decoded via the 
microphone in the 
phone using the 
phone's processor to 
decode the data 



A schematic outlining a system for closed captioning in cinemas using a mobile phone 
for the download of caption text, decoding of acoustic synchronisation signals and 
display of the caption text. 



